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DESCRIPTION 
Coding Device and Method, and Decoding Device and Method 

Technical Field 

This invention relates to a coding device and method for coding images, and a 
decoding device and method for decoding coded data. 

Background Art 

Conventionally, when coding digitized television signals, peripheral information 
of pixels to be transmitted is used for coding for the following reason. That is, an 
image generally has strong autocorrelation in a neighboring area, and it is more 
efficient*© ^se ^ata of^the ne^boring area incase of compression. 

- c Microscopically, however, tastrong correlation is found a flat part where no 
signal change occurs while little correlation can be found in an edge part of an image 
where the signal abruptly changes. 

In such case, conventionally, the strength of correlation is fully utilized for 
coding in a part where a strong correlation is found, and in an edge part, a 
corresponding quantity of information is allocated for carrying out coding, or coding 
is carried out within such a range that a visual masking effect can be obtained. 

Meanwhile, in the conventional coding, a corresponding quantity of information 
is allocated for carrying out coding in an edge part of an image. Therefore, the 



reduction in the quantity 
efficiency. 
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of information is limited, thus deteriorating the coding 



Disclosure of the Invention 

In view of the foregoing status of the art, it is an object of the present invention 
to provide a coding dev.ce and method which enables reduction in the quantity of 
infonnation and improvement in the coding efficiency of a signal value, by finding a 
pixel of the highest correlation even in an edge part and carrying out coding through 
random scan. 

It is another object of the present invention to provide a decoding device and 
method which enables easy decoding of an image that is coded and transmitted in a 
random scan order in accordance with the characteristics of the image. 

A coding device according- to -the present invention includes: an evaluation 
section for deciding, on the basis of the characteristics of an image signal having a 
plurality of pixel data, the coding order for the plurality of pixel data; and a coding 
section for coding the plurality of pixel data in the order decided by the evaluation 
section. 

A decoding device according to the present invention is adapted for decoding, 
from a plurality of coded pixel data generated by coding an image signal made up of 
a plurality of pixel data having a predetermined order in an order based on the 
characteristics thereof, the plurality of pixel data having the predetermined order. The 
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decoding devicei„cludes;aposition data extraction section forextractingpositiondata 
included in each of the plurality of coded pixel data; a level data extraction section for 
extracting level data included in each of the plurality of coded pixel data; and a 
conversion section for converting the level data of the plurality of coded pixel data to 
the predetermined order on the basis of the position data. 

A coding method according to the present invention includes: a step of deciding, 
on the basis of the characteristics of an image signal having a plurality of pixel data, 
the coding order for the plurality of pixel data; and a step of coding the plurality of 
pixel data in the order decided at the step of deciding. 

A decoding method according to the present invention includes; a step of 
extracting, on the basis of an image signal having a plurality of pixel data, the coding 
order for the plurality of pixel data; and a step of decoding the plurality of pixel data 
in the order extracted at the step of extracting. 

Brief Description of the Drawings 

Fig.l is a block diagram showing the structure of an image processing system, 

which is an embodiment of the present invention. 

Fig.2A is a signal distribution view showing biased distribution of a typical 
image signal in a color space of an RGB colorimetric system. 

Fig.2B is an infonnation distribution view in an address space. 

Fig.3 A is a view for explaining the concept of conventional scan. 
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Fig.3B is a view for explaining the concept of scan according to the present 
invention. 

Fig.4 is a view showing the format of pixel information. 

Fig.5 is a block diagram showing the detailed structure of an encoder in a 
coding device of the image processing system. 

Fig.6 is a flowchart for explaining optimization processing of the pixel 
transmission order carried out by the encoder shown in Fig. 5 . 

Fig.7 is a block diagram showing the detailed structure of an evaluation section 

constituting the encoder. 

Fig.8 is a block diagram showing the detailed structure of an evaluation function 

unit constituting the evaluation section. 

Fig:9^s^~block diagi^n showing the^detailed structure of a correlation 
discrimination unit constituting the evaluation,function unit. ~- 

Fig. 10 is a block diagram showing the structure of a decoder in a receiving 
device of the image processing system. 

Fig. 11 is a block diagram showing another specific example of the encoder 

shown in Fig.5. 

Fig. 12 is a view showing four pixels of a 2><2-pixel block used for explaining 
decimation processing carried out by the encoder shown in Fig. 1 1 . 

Figs. 13 A, 13B, 13C and 13D are views showing four patterns of decimation 
performed on the four pixels shown in Fig. 12. 
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Fig. 14 is a block diagram showing another specific example of the decoder 
shown in Fig. 10. 

Best Mode for Carrying Out the Invention 

A preferred embodiment of the present invention will now be described with 
reference to the drawings. In this embodiment, an image processing system 1 as 
shown in Fig. 1 is employed. The image processing system 1 includes a coding device 
2 for coding digitized pixel information and outputting coded data, a transmission 
medium 10 for transmitting the coded data outputted from the coding device 2, and a 
decoding device 6 for receiving the coded data transmitted by the transmission 
medium 10 an<l decoding the received data. 

With respect to a part where raster scan is disadvantageous to image correlation 
such as an edge part of an image of a typical television signal, the coding device 2 
searches for another candidate having stronger image correlation so as to carry out 
coding, rather than coding pixel information in the disadvantageous direction. In 
short, instead of employing a regular coding order, pixel information as a coding target 
is sequentially decided in a random direction, which is different from a raster scan 
direction decided on the basis of the correlation of pixel infonnation, in accordance 
with the characteristics and signal distribution of the image. 

In a typical color image signal, the signal level distribution of the image is 
biased to a certain extent as shown in Fig.2 A, for example, in a color space of an RGB 
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colorimetric system. On the other hand, when expressed in an address space, the 
image signal is uniformly distributed in a space such as a macroblock as shown in 
Fig.2B. Thus, the coding device 2 splits an image signal having a plurality of pieces 
of pixel information into a plurality of macroblocks, and coded and transmits level 
information of each pixel information in the macroblocks and position information in 
accordance with the characteristics and signal distribution of the image signal. 

The concept of the scan system according to the coding method of the present 
invention, employed by the coding device 2, will now be described with reference to 
Fig.3. In the case of block coding with respect to a block of a given size, pixel 
information in the block is usually coded in a raster scan order as shown in Fig.3A. 
On the contrary , in the coding method of the present invention employed by the coding 
device ^an optimum macroblock area is set so as to search for the next pixel 
information to be coded, and the pixel information is coded in the optimum order 
within the-macroblock: Therefore, as shown in Fig:3B, the pixel information in each 
macroblock is coded and transmitted in a random order. The optimum order for 
coding the pixel information may be decided in an entire frame or in an entire field. 

In the coding device 2 of Fig. 1 , digital pixel information inputted from an input 
tenninal IN T in a raster scan order is stored in memories 3a and 3b. These memories 
3a and 3b have a bank switching structure in which pixel information of each 
macroblock is read out from one memory while pixel information of each macroblock 
is written into the other memory. Therefore, a macroblock read section 4 can read out 
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pixel information of each macroblock from the memories 3a and 3b at different timing. 

The pixel information read out for each macroblock by the macroblock read 
section 4 is supplied to an encoder 5. The encoder 5 optimizes the transmission order 
for the pixel information in the macroblock and removes the redundancy, thus 
outputting coded pixel data. The coded pixel data from the encoder 5 is outputted to 
the transmission medium 1 0 via an output terminal OUT T . 

The coding by the encoder 5 will now be described. 

First, an example of the fonnat of pixel information is shown in Fig.4. As pixel 
information P, signal level information L of the pixel and position information A of the 
pixel are used. As the signal level information L of the pixel, R, G and B primary 
colorsare'consideredinthiscase 

sig^^tod^ited^pr^fference sign*! Cr may be considered. As the position 
informatidn A bf the P ixel,,addres S positions X and Y of a target pixel on the two- 
dimensional coordinate are considered. 

The encoder 5 splits each pixel information P in the macroblock into five 
components R, G, B, X and Y as shown in Fig.4, and codes and transmits the 
components. The encoder 5 finds the difference between pixel information Ps (Rs, Gs, 
Bs, Xs, Ys) and pixel candidate Pn (Rn, Gn, Bn, Xn, Yn) to be coded and transmitted 
next, and finds the sum E of absolute values of the difference as expressed by an 
equation (1). Using the sum E of absolute values as an evaluation value, the encoder 
5 decides the next pixel information Pn to be coded and transmitted so that the 
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minimum evaluation value is obtained. 

E = |Rn-Rs| + |Gn-Gs| + |Bn-Bs| + |Xn-Xs| + |Yn-Ys| 

• ••(I) 

After deciding the transmission order using the evaluation function of the 
equation (1), the encoder 5 differentially codes the next pixel infonnation Pn to be 
transmitted, and outputs the coded pixel data to the transmission medium 10. The 
encoder 5 will be later described in detail. 

As the transmission medium 10, a disc-like or tape-like recording medium may 
be used as well as a communication channel such as a network. 

The coded pixel data transmitted through the transmission medium 10 is 
inputted to the decoding device 6 via an input terminal IN R . A decoder 7 decodes the 
ad^e^mioTn^iioa^nmymdsto^s decoded values of the signal level infonnation 
at positions based on the address infonnation X and Y in memories 9a and 9b having 
a band switching structure. Then, a macroblock read section 8 reads out the signal 
level infonnation in the macroblock in the raster order from the memories 9a and 9b, 
and outputs the signal level infonnation from an output tenninal OUT R . 

The structure and operation of the encoder 5 and the decoder 7 will now be 
described in detail. 

The details of the structure of the encoder 5 are shown in Fig.5. This encoder 
5 includes an evaluation section 13 for evaluating the characteristics (strength of 
correlation between pixels) of an image signal using the evaluation value E of the 
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processing for selecting the transmission pixel information candidate Pn is carried out 
with respect to pixel information which is determined as not being transmitted yet at 
step S3, and is not carried out with respect to pixel information which is determined 
as being already transmitted. 

Then, at step S4, the encoder 5 uses the evaluation section 13 to evaluate the 
correlation with respect to the pixel information candidate Ps using the evaluation 
function of the equation (1). The equation (1) is adapted to find the difference 
between the pixel information Ps (Rs, Gs, Bs, Xs, Ys) to be transmitted and the pixel 
infonnation candidate Pn (Rn, Gn, Bn, Xn, Yn) to be transmitted next in the 
macroblock, and uses the sum of absolute value of the difference as the evaluation 

value TL - 

Subsequentiy; at step S5, the-encoder 5 determines whether the evaluation value 
E of the evaluation^funcrion ofithe equation (1) is the minimum or not. If the 
evaluation value E is the minimum value, the processing goes to step S6 and the value 
of pixel infonnation in a minimum value buffer, later described, provided inside the 
evaluation section 13 is rewritten. If the evaluation value E is not the minimum value, 
rewrite is not carried out. The encoder 5 repeats the minimum value search processing 
up to this step with respect to all the pixel infonnation in the macroblock (step S7). 

On completion of the search processing with respect to all the pixel information 
in the macroblock at step S7, the encoder 5 transmits the pixel information in the 
minimum value buffer as the next pixel infonnation to be transmitted (step S8). 
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By repeating the processing of steps S2 to S8 until all the pixels in the 
macroblock are transmitted (step S9), the encoder 5 decides the next pixel information 
to be transmitted. As a matter of course, the encoder 5 may decide, at this point, the 
next pixel information to be transmitted and may transmit it at any time. 

The principle of the search processing with respect to all the pixels in the 
encoder 5 (corresponding to step S7) will now be described using an exemplary 
hardware structure as shown in Fig.7. 

It is assumed that the hardware structure has a plurality of evaluation function 
units for computing the evaluation function of the equation (1) from the level 
information R, G, B and the position information X, Y of the pixel infonnation so as 
to find the evaluation value E. • 

When a clock rate for processing eachjpixel is used, with respect to n pieces of 
pixel infonnation from an input terminal 2 1 5 an evaluation function unit 22 n ., repeats 
calculation of the evaluation value E in accordance with the evaluation function of the 
equation (1) for n-1 times, and decides pixel infonnation which realizes the lriinimum 
evaluation value E, as the pixel to be transmitted next to the initial transmission pixel 
infonnation. 

An evaluation function unit 22 n . 2 repeats calculation of the evaluation value E 
in accordance with the evaluation function of the equation ( 1) for n-2 times, excluding 
the two pieces of pixel infonnation for which the transmission order is already decided 
by the calculation at the evaluation function unit 22„.„ and thus decides the third pixel 
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information to be transmitted. 

Then, the calculation of the evaluation function of the equation (1) is repeated 
for n-3 times, n-4 times, once, until an evaluation function unit 22, decides the last 
pixel to be transmitted in the macroblock. 

The evaluation function units 22 n . l5 22 n . 2 , 22, are connected with 
transmission flag memories 23 n .„ 23„. 2 , 23„ respectively, in which a flag indicating 
"transmission completed" or "transmission not completed" is stored for every 8x8 
pixels. 

The detailed structure of the evaluation function units 22 n . l5 22 n ; 2 , 22, is 
shown in Fig.8. On the assumption that the components of the previously transmitted 
pixel information are referred to as : former values (including the initial transmission 
pixel tmforination) ^*ile thetcp^ponents of the next pixel information to be 
transmitted are referred to as latter values (candidate values of transmission pixel 
information), the correlations between the former values R„ G„ B„ X„ Y, and the 
latter values R 2 , G 2 , B 2 , X 2 , Y 2 are discriminated by correlation discrimination units 
25 R , 25 G , 25b, 25 x and 25 v> respectively. In the correlation discrimination unit 25, for 
example, with respect to R as shown in Fig.9, the difference R 2 -R, between the fonner 
value R, and the latter value R 2 is found by a difference section 3 1, and the absolute 
value of the difference |R 2 -R,| is found by an absolute value section 32. Then, the 
absolute value of the difference |R 2 -R,| and the latter value R 2 are outputted. 

The absolute values of the differences of the respective components from the 
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correlation discrimination units 25 R , 25 G , 25 B , 25 x and 25 Y are supplied to an adder 26. 
The latter values of the respective components are supplied to latches 29 R , 29 G , 29 B , 
29 x and 29 v . 

The addition result obtained by the adder is equal to the evaluation value E as 
follows. 

IRa-Rjl + |G 2 -G,| + P 2 -Bil + IXrXd + |Y 2 -Y,| 
The addition result is sent to a comparator 28. The comparator 28 compares an 
evaluation value from a minimum value buffer 27 which stores the minimum 
evaluation value up to this point, with the current evaluation value obtained as the 
result of addition. If the current evaluation value (addition result from the adder 26) 
is smaller than the evaluation value stored k^e,#iimum value buffer 27, a reset 
output>is sent to theTinnimum value buffer 2Xancl the latches 29 R , 29 G , 29 B , 29 x and 
29 Y so as to reset the values of the buffer anddatches. Therefore, in the latches 29 R , 
29 29 29 • and 29 v the pixel information candidate used for calculating a new 
evaluation value, that is, the latter values R 2 , G 2 , B 2 , X 2 , Y 2 are stored, respectively. 
After the search processing with respect to all the pixel information in the macroblock 
is completed, the last pixel information Pn (Rn, Gn, Bn, Xn, Yn) stored in the latches 
29 R , 29 G , 29 B , 29 x and 29 Y is transmitted. 

The pixel information Pn (Rn, Gn, Bn, Xn, Yn) thus transmitted from the 
evaluation section 13 is supplied to subtracters 15 R , 15 G , 15 B , 15 x , 15 Y . The 
subtracters 15 R , 15 G , 15 B , 15 x , 15 Y calculate difference values D R , D G , D B , D x , D Y 
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between the already transmitted pixel information Ps (Rs, Gs, Bs, Xs, Ys) and the next 
pixel information to be transmitted Pn (Rn, Gn, Bn, Xn, Yn) stored in latches 14 R , 14 G , 
14 Bj 14 x , 14 y , and send the difference values to differential coders 16 R , 16 G , 16 B , 16 x , 
16 Y of the differential coding section 16, respectively. The differential coders 16 R , 
16 G , 16 B , 16 x , 16 Y differentially code the difference values D R , D G , D B , D x , D Y , 
respectively. As a differential coding method, there is employed DPCM for re- 
quantizing the difference value, or a coding method using Huffman coding with 
optimization of the frequency of the difference value. 

The coded values of the differences of the respective components from the 
difference coding section 16 are multiplexed by the multiplexing section 17 and 
transmitted to the decoding;device 6 through the. transmission medium 10. 

The=decodeF 7 shown in Fig. 1 will now be described in detail with reference to 

Fig&O^This decoder^ carries out the decoding method of the present invention. The 
decoding method is adapted for decoding an image which is coded and transmitted in 
a random scan order in accordance with the characteristics of the image. In this 
method, pixel information of the transmitted image is decoded, and the image signal 
is read out in a raster scan order on the basis of the decoded pixel information. 

To carry out the decoding method, the decoder 7 includes a splitting section 42 
for splitting the coded value of the difference of the pixel information multiplexed in 
the encoder 5 into coded values of differences of the respective components, a 
differential decoding section 43 for decoding the differences from the coded values of 
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differences of the respective components split by the splitting section 42, adders 44 
and latches 45 constituting a component decoding section for obtaining component 
values of the pixel information from the differential decoding output from the 
differential decoding section 43, and macroblock memories 46a and 46b in which the 
level information R, G, B of the pixel information is written on the basis of the address 
information X, Y obtained by the component decoding section and from which the 
pixel infonnation is subsequently read out in an ordinary scan order. The address for 
reading at the macroblock memories 46a and 46b is counted by an address counter 47 
as an address following the ordinary scan order. 

The following is the flow of operation of the decoder 7. That is, the splitting 
section 42 splits the differential coding value of the multiplexed components randomly 
transmitted thereto from -the .input terminal 6, and supplies the split values to 
differential-decode^^ section 43, 

respectively. 

The difference values of the respective components decoded by the differential 
decoders 43 R , 43 G , 43 B , 43 x , 43 Y are supplied to the adders 44 R , 44 G , 44 B , 44 x , 44 Y 
constituting the component decoding section, and are added to latch addition outputs 
from the latches 45 R , 45 G , 45 B , 45 x , 45 Y . The respective component decoding outputs 
from the component decoding section are supplied to the macroblock memories 46a 
and 46b having a bank structure. Since the address counter 47 reads out, by raster 
scan, the address infonnation used for random scan as described above, an image 
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signal that is written in a random scan order is converted to an image signal in a raster 
scan order and then outputted from the macroblock memories 46a and 46b. 

Thus, in the above embodiment, since a pixel of high correlation is searched for 
so as to carry out coding by raster scan coding with respect to an edge part or the like 
where no correlation can be found, the coding efficiency of the signal value is 
significantly high. Although the quantity of information is increased as address 
information which is not necessary for raster scan is sent, the quantity of information 
of the signal value can be reduced to a greater extent and therefore the coding 

efficiency is improved as a whole. 

In the coding device 2 of the image processing system shown in Fig.l, an 

encod^sh^^ - P lace of the enC ° der 5 " ^ 5 ° 

differs^fromthe*ncoder,5 in that animation section 51 is provided on the stage 

before the evaluation section 13. 

The decimation section 5 1 .reduces the signal level information and address 
information of the pixel information. The principle of the decimation section 51 will 
now be described with reference to Figs. 12 and 13 A to 13D. The four pixels of a 2x 2 - 
pixel block shown in Fig. 12 are taken into consideration, and four patterns of pixel 
densities of Fig. 13 A to 13D are adaptively employed with respect to the signal 
distribution of the four pixels. The four pixels have pixels values a, b, c, d, 
respectively. 

It is now assumed that TH represents a threshold value. In the pattern 1 of 
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Fig. 13A, the pixel values are replaced by (a+b+c+d)/4 when |a-b| < TH, |b-c| < TH, |c- 
d| < TH, |d-a| < TH, |a-c| <TH and |b-d| < TH are all satisfied. In the pattern 2 of 
Fig. 1 3B, the pixel values are replaced by (a+c)/2 and (b+d)/2 when only |a-b| <TH and 
|c-d| < TH are satisfied. In the pattern 3 of Fig. 13C, the pixel values are replaced by 
(a+b)/2 and (c+d)/2 when only |a-c| <TH and |b-d| < TH are satisfied. In the pattern 
4 of Fig.l3D, the original pixel values are maintained when none of the above 
conditions is met. 

By using the encoder 50 having this decimation section 51 for the coding 
device, the quantity of information is reduced. 

Fig. 14 shows the structure of a decoder 53 which is necessary when the encoder 
50 is used. On the image signal read out from the macroblock memories 46a and 46b, 
interpolation processing using a line memory 54 must be performed by a pixel address 
interpolation section -55:; The interpolation output is outputted from an output terminal 
56. 

In the coding device 1 , the random scan order may be optimized with respect 
to pixels for each macroblock in a frame image, or the random scan order may be 
optimized with respect to pixels for each macroblock in a field image. Moreover, a 
macroblock in the direction of time base may be used as a unit. The macroblocks need 
not be sent sequentially and the difference of the leading address of the macroblocks 
may be sent. 
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Industrial Applicability 

According to the present invention, since a pixel of high correlation is searched 
for so as to carry out coding even in an edge part, the quantity of information can be 
reduced and the coding efficiency of the signal value can be improved. Also, an image 
which is coded and transmitted in a random scan order in accordance with the 
characteristics of the image can be decoded with a simple structure. 



